With the development of data processing techniques, data processing is no longer limited to serial processing, but it is feasible to perform parallel processing across multiple nodes in a distributed processing system. For an application involving big data processing, distributed parallel processing may greatly increase the data processing efficiency and further provide more support for real-time data processing.
Real-time data processing may be applied to multiple application environments. For example, real-time monitoring has become an important application hotspot nowadays. A large amount of video stream data will be generated during monitoring important areas such as city roads, main transport hubs and so on. For another example, large amounts of real-time monitoring stream data (e.g., sampled at intervals of 1 second) on temperature, humidity and so on will also be generated in fields of environment monitoring, production line monitoring, etc. Since the amounts of information carried by these stream data vary, it is highly possible that the workload rockets at a certain moment and extra computing resources are required. At this point, it becomes a focus of research regarding how to dynamically process in parallel stream data more effectively.